Systems and methods for maintaining virtual component checkpoints on an offload device

ABSTRACT

Generally described, aspects of the present disclosure relate to offload device virtual component checkpointing for fast recovery from virtual component software crashes by storing virtual component state configuration information and input/output (I/O) request identification information in non-volatile memory of a physical computing device physically separate from the offload device. In the event of a software crash of a virtual component, the crashed virtual component may be rebooted and reconfigured in accordance with the virtual component state configuration information and I/O request identification information stored in the non-volatile memory of the physical computing device.

INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS

Any and all applications for which a foreign or domestic priority claimis identified in the Application Data Sheet as filed with the presentapplication are incorporated by reference under 37 CFR 1.57 and made apart of this specification.

BACKGROUND

Generally described, computing devices utilize a communication network,or a series of communication networks, to exchange data. Companies andorganizations operate computer networks that interconnect a number ofcomputing devices to support operations or provide services to thirdparties. The computing systems can be located in a single geographiclocation or located in multiple, distinct geographic locations (e.g.,interconnected via private or public communication networks).Specifically, data centers or data processing centers, herein generallyreferred to as a “data center,” may include a number of interconnectedcomputing systems to provide computing resources to users of the datacenter. The data centers may be private data centers operated on behalfof an organization or public data centers operated on behalf, or for thebenefit of, the general public.

To facilitate increased utilization of data center resources within thedata centers, virtualization technologies may allow a single physicalcomputing device to host one or more instances of virtual machines thatappear and operate as independent computing devices to users of a datacenter. With virtualization, software applications running on thephysical computing device can create, maintain, delete, or otherwisemanage virtual machine instances in a dynamic manner.

Use of the data centers in increasing numbers has created increaseddemand for the computing resources. Even with virtualizationtechnologies, the number of available resources that can be provided tothe virtual machines is limited, at least in part, by the softwareapplications managing the virtual machine instances in the physicalcomputing devices. The cost associated with changing the existinghardware resources for better hardware components can be a considerableexpense.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisdisclosure will become more readily appreciated as the same becomebetter understood by reference to the following detailed description,when taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram depicting a physical computing device with anoffload device and a plurality of virtual machine instances.

FIG. 2A illustrates an embodiment of a state flow diagram depicting theconfiguration of virtual machine instances on the physical computingdevice and virtual components on an offload device utilizing a controlplane manager.

FIG. 2B illustrates an embodiment of a state flow diagram depicting theconfiguration of virtual machine instances on the physical computingdevice and virtual components on an offload device by a virtual machinemonitor.

FIG. 3A illustrates an embodiment of a state flow diagram depictingindirect communication between a virtual machine instance and a virtualcomponent via the virtual machine monitor.

FIG. 3B illustrates an embodiment of a state flow diagram depictingdirect communication between a virtual machine instance and a virtualcomponent.

FIG. 4 illustrates a flow diagram of a routine for determining aconfiguration of a virtual environment on a physical computing deviceand an offload device by a control plane manager.

FIG. 5 illustrates a flow diagram of a routine for the configuration ofa virtual environment on a physical computing device and an offloaddevice.

FIG. 6 illustrates a flow diagram of a routine for indirectcommunication between the host computing device and the virtual devicevia the virtual machine monitor.

FIG. 7A is a block diagram illustrating a process of storing virtualcomponent state configuration information characterizing an initialvirtual component state configuration of the virtual component forvirtual component checkpoints.

FIG. 7B is a block diagram illustrating a process of storing virtualcomponent state configuration information characterizing an updatedvirtual component state configuration of the virtual component forvirtual component checkpoints.

FIG. 7C is a block diagram illustrating a process of storing I/O requestidentification information for virtual component checkpoints.

FIG. 8 is a block diagram illustrating a process of retrieving virtualcomponent state configuration information and I/O request identificationinformation for virtual component checkpoints.

FIG. 9 is a flow diagram illustrating a process of storing virtualcomponent state configuration information and I/O request identificationinformation for virtual component checkpoints

FIG. 10 is a flow diagram illustrating a process of retrieving virtualcomponent state configuration information and I/O request identificationinformation for virtual component checkpoints.

DETAILED DESCRIPTION

Generally described, a physical computing device can be configured tohost a number of virtual machine instances. Specifically, such physicalcomputing devices can execute a virtual machine monitor can be used tomanage multiple aspects of virtual machine instances. Such a virtualmachine monitor may often be referred to as a “hypervisor.” The virtualmachine monitor can associate and manage three primary virtualizedresources to instantiated virtual machine instances, namely, virtualprocessing resources, virtual memory resources, and virtual input/output(I/O) resources, collectively or individually referred to as virtualcomponents.

Although referred to as virtual components, the instantiation andoperation of the virtual components requires computing resources of thephysical computing device to implement. Generally, the virtual machinemonitor will manage virtual components for each instantiated virtualmachine instance on the physical computing device. As a result, physicalcomputing resources are consumed to support the instantiated virtualcomponents of each instantiated virtual machine instance and reduce theavailability of the physical computing device resources for instantiatedvirtual machine instances or additional virtual machine instances.

The present application relates to systems and methods for the managingthe instantiation and execution of virtual machines instances using aphysical computing device and an offload device. In accordance with anillustrative embodiment, the offload device corresponds to anindependent computing device that includes physical computing resources(e.g., processor and memory) separate from the physical computingresources associated with the physical computing device hosting theinstantiated virtual machine instances. The offload device can beconnected to the physical computing device via an interconnectinterface. The interconnect interface can be a high speed, highthroughput, low latency interface such as a Peripheral ComponentInterconnect Express (PCIe) interface. The offload device can be used tocontrol the virtual machine monitor and emulate certain virtualcomponents associated with the instantiated virtual machine instances,thereby decreasing the need to utilize physical computing resources inthe execution of the instantiated virtual machine instances.

In accordance with an illustrative embodiment, the offload device can beused to instantiate virtual machines on the physical computing device.For example, the offload device can receive a command from a controlplane manager via a network interface integrated into the offload deviceor a management domain and instruct the virtual machine monitor tolaunch virtual machines. In addition, the virtual machine monitor canprovide resource information regarding the physical computing to thecontrol plane manager via the offload device. The control plane managercan determine based on the resource information, such as the specifichardware configuration, and other information, such as the anticipateduse of the physical computing device, the configuration of virtualmachine instances on the physical computing device and virtualcomponents on the offload device. The control plane manager can provideinstructions to the virtual machine monitor to instantiate virtualmachine instances in the determined configuration and instruct theoffload device to instantiate virtual components in the determinedconfiguration. The virtual machine monitor can provide mapping of theinstantiated virtual components on the offload device such that thevirtual machine instances can recognize and communicate with the virtualcomponents through the interface bus.

In accordance with another illustrative embodiment, the virtual machinemonitor can be configured to instantiate virtual machine instances onthe physical computing device and instantiate respective virtualcomponents on the offload device. The configuration of the virtualmachine instances can determine the virtual components that areinstantiated on the offload device on behalf of the instantiated virtualmachine instances. The virtual machine monitor can also provide mappingof the instantiated virtual components on the offload device such thatthe virtual machine instances can recognize and communicate with thevirtual components through the interface bus.

In accordance with an illustrative embodiment, the instantiated virtualI/O components on the offload device are configured to execute orprocess at least a portion of the I/O requests generated by theinstantiated virtual machine instances. Illustratively, the virtualmachine instances can communicate one or more I/O requests with theinstantiated virtual I/O components on the offload device. In someaspects, the instantiated virtual machine instances may communicatedirectly with the virtual I/O components via the interface bus. In otheraspects, the instantiated virtual machine instances can communicateindirectly with the virtual I/O components via the virtual machinemonitor. In this aspect, the virtual machine monitor can use atranslation table to route the I/O request to the virtual component. Thetype of communication can be based, at least in part, on communicationprotocols associated with the virtual I/O components or other criteria.

Generally described, aspects of the present disclosure relate tomaintaining and updating state information associated with virtualcomponents hosted on an offload device. The state information can beused for recovery of the virtual components after the occurrence of atriggering event such as a software crash. In accordance with anillustrative embodiment, the offload device corresponds to anindependent computing device that includes physical computing resources(e.g., processor and memory) separate from the physical computingresources associated with the physical computing device hosting theinstantiated virtual machine instance. The offload device can be subjectto triggering events that are isolated from the physical computingdevice. A triggering event could cause the failure and subsequentrebooting of the offload device. Such a failure of the offload devicewould reset the virtual components hosted on the offload device. In suchan event, virtual machine instances, which would otherwise be isolatedfrom the operation of other virtual machine instances, would lose dataassociated with the virtual components hosted on the offload device.

In accordance with an illustrative embodiment, state configurationinformation can be used for recovery of virtual components after atriggering event, such as a software crash. Specifically, this may beaccomplished by storing state configuration information and input/output(I/O) request identification information associated with the virtualcomponent in memory on the physical computing device that is separatefrom the offload device. The state configuration information can beupdated every time an event changes the state configuration of theevent. The I/O request identification information can trackidentification information associated with transitory I/O requests thatare pending for the virtual component. The information, including bothstate configuration information and I/O request identificationinformation, stored for each virtual component can be referred to as avirtual component checkpoint.

In the presence of a triggering event, such as a software crash of avirtual component, the virtual component may be rebooted andreconfigured in accordance with the state configuration information andthe I/O request identification information stored in the memory of thephysical computing device. The reconfigured virtual component can beconfigured to have the same operational state the virtual component hadprior to the triggering event. Thereby, the virtual component canrecover from the triggering event without requiring re-initialization ofthe virtual component by the virtual machine instance. When a triggeringevent occurs a virtual machine monitor can be freeze operation of thevirtual machine instances until the reconfiguration of the virtualcomponents is complete. After reconfiguration the virtual components canuse the I/O request identification information to request that thepending I/O requests be retransmitted for processing. In this manner,the virtual machine instances can resume operation as normal. The entirereconfiguration process can be opaque to the virtual machine instancessuch that it can appear that the triggering event did not occur.

While specific embodiments and example applications of the presentdisclosure will now be described with reference to the drawings, theseembodiments and example applications are intended to illustrate, and notlimit, the present disclosure. Specifically, while various embodimentsand aspects of the present disclosure will be described with regard toillustrative components of host computing device, one or more aspects ofthe present disclosure can be applied with regard to different types orconfigurations of physical computing devices or combinations thereof.

FIG. 1 illustrates an embodiment of a physical computing device 100configured to host virtual machine instances 120 and interact with anoffload device 130. The physical computing device 100 includes one ormore processing units 102, such as one or more CPUs. The physicalcomputing device 100 includes memory 104, which may correspond to anycombination of volatile or non-volatile computer-readable storage media.The memory 104 may store information which includes various programs,program data, and other modules. The programs stored in the memory caninclude a virtual machine monitor application 110 that can manage thevirtual machine instances (e.g., by allocating memory to each virtualmachine instance and scheduling virtual processors to run on physicalprocessors). The physical computing device 100 can correspond to a widevariety of devices, such as servers, that include a wide variety ofsoftware and hardware components and configurations. The physicalcomputing device 100 can include a local data store (not shown), or beconfigured to communicate with a data store over a network (not shown).

The physical computing device 100 is capable of hosting a plurality ofvirtual machine instances 120. The virtual machine monitor 110 canmanage virtual machine instances 120 hosted on the physical computingdevice 100. Illustratively, the management of the virtual machineinstances 120 can include the instantiation of the virtual machineinstance and the instantiation of virtual components utilized in theexecution of the virtual machine instance. Additionally, as will beexplained in greater detail, the management of the virtual instances canfurther include the management of interaction between the virtualmachine instances and the offload device 130 In the illustratedembodiment, the physical computing device 100 includes two instantiated,or hosted, virtual machine instances 120, virtual machine instance “A”and virtual machine instance “B”. One skilled in the relevant art willappreciate, however, that the physical computing device 100 can host anynumber of virtual machine instances and is not limited to the hosting ofthe two virtual machine instances illustrated in FIG. 1.

With continued reference to FIG. 1, the offload device 130 can beoperably coupled to the physical computing device 100 via theinterconnect interface 106. The interconnect interface 106 can refer toa physical communication interface on the physical computing device 100.The interconnect interface 106 can be an electrical communicationinterface, an optical communication interface or other type ofinterconnect interface known in the art. The interconnect interface 106can be configured to provide communications between components hosted onthe offload device 130 with the virtual machine instances 120 hosted onthe physical computing device 100. Illustratively, the configuration ofthe interconnect interface can be optimized based on specific criteria,such as low latency, high speed, and high bandwidth, among others. Insome embodiments, the interconnect interface can correspond to a highspeed serial computer expansion bus, such as a Peripheral ComponentInterconnect Express (PCIe) bus. However, one skilled in the relevantart will appreciate that the interconnect interface may incorporatealternative or additional standard interconnect interfaces well known tothose skilled in the art of computer architectures.

In an example embodiment, the offload device 130 is a computing device,or partial computing device, operably coupled to the physical computingdevice 100. In some embodiments, the offload device 130 is physicallycoupled to the physical computing device 100 as well. The offload device100 includes one or more processing units 132, such as one or more CPUs.The offload device 130 includes memory 134, which may correspond to anycombination of volatile or non-volatile computer-readable storage media.The memory 134 may store information which includes various programs,program data, and modules, such as a management module 138 and a deviceemulation module 139. The management module 138 can management componentcan be configured to determine the type of virtual components toinstantiate based on configuration information for the virtual machineinstance. The device emulation module 139 can be configured to performthe emulation and instantiation of the virtual components on the offloaddevice 130. The processor 132 and memory 134 of the offload device 130are separate from the processor 102 and memory 104 of the physicalcomputing device 100. The offload device 130 can include a local datastore (not shown), and/or be configured to communicate with a data storeover a network (not shown).

The offload device can include a network interface controller (NIC) 136.The offload device can be in communication with a control plane manager150 (illustrated in FIG. 2A) via a network. The offload device can beconfigured act as an intermediary for providing instructions from thecontrol plane manager 150 to the virtual machine monitor 110, which willbe explained in greater detail below with respect to FIG. 2A.

As will be explained in greater detail below, the offload device 130 canhost and emulate one or more virtual components that are used by theinstantiated virtual machine instances substantially independent of oneor more physical computing device 100 resources. The offload device 130can be dynamically reconfigurable, such that the virtual components 140can be removed, changed, added, or otherwise reconfigured to address theconfiguration of the virtual machine instances 120 on the physicalcomputing device 100. Accordingly, the offload device 130 would use atleast a portion of the physical computing resources on the offloaddevice to carry out the function of the instantiated virtual components.By way of example, operations executed on the offload device 130 can becarried out using the computing resources (e.g., processor 132 andmemory 134) without requiring the usage of the physical computingdevice's 100 computing resources (e.g., processor 102 and memory 104).

In accordance with an illustrative embodiment, at least some portion ofthe virtualized components hosted on the offload device 130 correspondto virtual I/O components configured to execute I/O functionality onbehalf of instantiated virtual machine instances. As illustrated in FIG.1, the offload device 130 can include virtual I/O component groups 142.Each virtual I/O component groups 142 corresponds to a virtual machineinstance 120 on the physical computing device 100. In the illustratedembodiment the offload device 130 includes virtual I/O component group Ais associated with virtual machine instance A and virtual I/O componentgroup B associated with virtual machine instance B. Each virtual I/Ocomponent group includes a plurality of virtual components 140

Generally described, the virtual machine monitor 110 executed on thephysical computing device 100 is configured to manage various aspectsassociated with instantiated virtual machines instances. In anembodiment, the management operations can be split between the virtualmachine monitor and a management domain, such as a Domain-0, that runson physical computing device 100. In yet another embodiment, all orportions of the programs that run within Domain-0 can instead run on theoffload device 130. The virtual machine monitor 110 can be executeddirectly on the physical computing system hardware. The virtual machinemonitor can function like a host operating system for the physicalcomputing device 100. The virtual machine monitor 110 can control thehardware of the physical computing device 100 and manage and configurevirtual machine instances 120 running on the physical computing device100. The virtual machine monitor 110 can implement management functionsthat provide direct access to the hardware of the physical computingdevice 100.

To support hosted virtual machine instances, the virtual machine monitor110 can instantiate guest domains on the physical computing device 100for each virtual machine instances 120 by allocating the guest domainsmemory and time on the physical CPUs. As previously described, theallocated virtual resources include three primary virtualized resourcesthat are utilized by the instantiated virtual machine instances, namely,virtual processing resources, virtual memory resources, and virtual I/Oresources. In some embodiments, the configuration of the virtual machineinstances 120 can be determined by a control plane manager 150 asdescribed in greater detail in FIG. 2A. In some embodiments, the virtualmachine monitor 110 can determine the configuration of the virtualmachine instances as described in greater detail in FIG. 2B.

Each virtual machine instance 120 is provisioned virtual resources thatare implemented by the physical computing resources of the physicalcomputing device 100. For example, a virtual machine instance can beallocated a virtual processing resources 122 and virtual memoryresources 124 that represent logically provisioned allocations ofunderlying computing resources of the physical computing device (e.g.,processor 102 and memory 104). Some of the virtual resources, such asvirtual I/O resources, can be offloaded to the offload device 130. Theconfiguration of the virtualized resources for each virtual machineinstance 120 can vary. For example, virtual machine instance A andvirtual machine instance B can have different allocations of virtualizedcomputing resources.

The virtual machine instances 120 may be provisioned to provide avariety of different desired functionalities depending on the needs of adata center or client. Examples of the types of desired functionalitycan include, but are not limited to: database management, serving ordistributing data or content (e.g., Web servers), managing loadbalancing or network resources, managing network connectivity orsecurity, providing network addressing information, managing client orserver redirection, or other functionalities. In some embodiments, thevirtual machine instances 120 may be provisioned to implement portionsof a hosted network or to simulate one or more components of a hostednetwork. Illustratively, the virtual machine instances 120 may beconfigured to provide specific functionality associated with thecomponents of a hosted network or simulation of the components of thehosted network. The virtual machine instances 120 may be provisionedgenerically when a desired functionality is not specified or isotherwise not available.

As previously describe, aspects of the present application relate to thehosting of virtual I/O components on the offload device 130 in a mannerthat reduces the execution of I/O functionality by the hosted virtualresources on the physical computing device 100. Each virtual machineinstance 120 is associated with virtual components 140 grouped intovirtual I/O component groups 142. The virtual machine monitor 110 isresponsible for the provisioning of virtual I/O component groups 142 foreach of the virtual machine instances 120. The virtual components 140can be logically grouped based on their association with a virtualmachine instance 120. The virtual machine monitor 110 can assign memoryaddress ranges to virtual components within the memory allocated tovirtual machine instances. For example, in FIG. 1, virtual I/O componentgroup A is associated with virtual machine A and the virtual I/Ocomponent group B is associated with virtual machine B. In someembodiments, the virtual components 140 can be provisioned and emulatedusing the computing resources (e.g., processor 132 and memory 134) ofthe offload device 130. For example, the offload device 130 can run oneor more programs that simulate the functions of hardware components thatwould be typically found on a motherboard of a computer system.

The virtual components 140 represent a set of virtual functions that canbe implemented by a virtual machine instances 120. The virtualcomponents 140 can provide virtual I/O functions that emulate the I/Ofunctions of hardware computing devices found in a physical computingdevice. For example, the virtual components can correspond to I/O devicetypes such as the real time clock (RTC), storage controller, networkinterface controller (NIC), programmable interrupt controller (PIC),peripheral component interconnect (PCI) bus, disk controller, SCSIcontroller, floppy drive, keyboard and mouse ports, monitor ports,serial ports, keyboard controller, ISA bus, and other I/O devices. Thevirtual components 140 are sometimes referred to as virtual devices. Ina virtual computing environment not every function needs to bevirtualized for every machine. The virtual machine monitor 110 candetermine which I/O devices need to be virtualized based on theconfiguration of the virtual machine instance 120.

In addition to the functionality implemented by the virtual I/Ocomponents, the various virtual components 140 can be configured as tothe specific communication protocols utilized by the instantiatedvirtual machines to access the I/O functionality. More specifically, inone embodiment, some of the virtual I/O components may correspond to aPort I/O communication protocol (“Port I/O virtual components”) andother virtual I/O components may correspond to a memory-managed I/O(MMIO) communication protocol (“MMIO virtual components”).

Port I/O virtual components can require specific communication protocolsfor communications between a virtual machine instance 120 and a Port I/Ovirtual component. In this embodiment, the virtual machine monitor 110can function as an intermediary to handle the communication protocolsrequired for the communication between the Port I/O virtual componentsand the virtual machine instance 120. The virtual machine monitor 110can include a translation table that is used for handling communicationbetween the virtual machine instance and port I/O virtual components140, some MMIO components also use a translation table. The translationtable can include a table for translating requests to and from thevirtual machine instance 120 in order to route the requests to thecorrect addresses assigned to the virtual components 140. Additionaldetails regarding the communication between virtual machine instances120 and virtual components 140 that utilize the virtual machine monitor110 are described below in associated with FIG. 3A.

The MMIO components can be configured such that the virtual machineinstance 120 can communicate directly with the virtual component 140 bycommunicating with the memory addresses assigned to the virtualcomponent 140. Additional details regarding direct communication betweenvirtual machine instances 120 and virtual components 140 described belowin associated with FIG. 3B.

The physical computing device 100 can be part of a network that includesmultiple physical computing devices 100. One skilled in the relevant artwill appreciate that the network is logical in nature and can encompassphysical computing devices 100 from various geographic regions.Additionally, the network can include one or more physical computingdevices 100 that do not host virtual machine instances 120.

The physical computing devices can be managed by a centralizedmanagement system, such as illustrated by the control plane manager 150in FIG. 2A. The control plane manager 150 can be configured to managethe operation and configuration of the physical computing devices 100 onthe virtual network as well as select computer systems to host virtualmachines and send launch instructions to the offload device 130 or amanager program that runs in Domain-0. The control plane manager 150 candetermine configurations, operating parameters, resource allocationswithin virtual machine instances, for each physical computing devicewithin a virtual network. In some embodiments, the managements systemcan comprise a plurality of control plane managers that controldifferent allocations of physical computing devices. The control planemanager 150 can be in communication with the physical computing devices100 through the offload device 130. In some embodiments, the controlplane managers can communicate directly with the offload device 130and/or the physical computing devices 100.

FIG. 2A illustrates a block diagram depicting the configuration ofvirtual machine instances 120 on the physical computing device 100 andvirtual components 140 on the offload device 130 by a control planemanager 150. The control manager 150 can be in communication with thephysical computing device 100 via the offload device 130. The functionsdescribed in association with FIG. 2A can be performed in a differentsequence, can be added, merged, or left out altogether (e.g., not alldescribed acts or events are necessary for the practice of thealgorithms). Moreover, in certain embodiments, functions, acts or eventscan be performed concurrently. For example, the operations associatedwith (6), (7) and (8) can occur in a different order or can occurconcurrently.

At (1), the virtual machine monitor 110 can determine the availableresources of the physical computing device. The available resources caninclude information such as the available computing resources, such asprocessor type, memory configuration, hardware component configuration,versioning information, and other information that identifies theresources of the physical computing device. The resource information caninclude current operating conditions, computer resource utilizationinformation associated with the current configuration of the physicalcomputing device 100. In some embodiments, the resource information caninclude resource information associated with the offload device 130. Theresource information can be gathered based on a request provided by theoffload device or the control plane manager. In some embodiments, thevirtual machine monitor may be configured to provide the resourceinformation based on a triggering condition, such as during a portion ofthe boot process. In some embodiments, the virtual machine monitor 110can be configured to periodically transmit the resource information.

In some embodiments, the resource information associated with a physicalcomputing device 100 can be stored in a data store configured formaintaining the resource information associated with each physicalcomputing device. The data store can also store the current operationalstate of the physical computing device, such as the instances that arerunning on the physical computing device 100.

At (2) the virtual machine monitor sends the resource information to theoffload device. After receipt of the resource information, at (3) theoffload device can send the resource information to the control planemanager. In some embodiments, the resource information provided by thevirtual machine monitor 110 can be supplemented with resourceinformation associated with the offload device 130. In such instances,the resource information may be sent together with the virtual machinemonitor resource information or sent separately. In some embodiments,the offload device 130 can periodically send information to the controlplane manager 150 to provide an update regarding the status of instancesthat are operating on the physical computing device 100.

At (4), the control plane manager can determine the configuration of thephysical computing device 100 and the offload device 130. Thedetermination can be based, in part, on the received resourceinformation and information independent of the resource information. Forexample, the control plane manager 150 can also base the configurationon other considerations, such as client specifications, theconfigurations of other physical computing devices 100, such asclustered computing devices, or other considerations independent of theresource information associated with the physical computing device. Insome instances the control plane manager can be configuring a newphysical computing device 100, updating an existing configuration aphysical computing device 100, adding/removing virtual instances, and/orperforming other management function associated with the physicalcomputing device 100.

The control plane manager 150 can determine the configuration of theallocation of the virtual machine instances on the physical computingdevices and virtual components on the offload devices. As part of theconfiguration of the physical computing device 100, the control planemanager 150 can determine the virtualized hardware resources allocatedto each virtual machine instance 120. A virtual machine instance 120 canhave a specific configuration according to the computing capacity of thevirtual machine instance. This can be based on the resource information,requirements of a customer, the system, number of instances operating ona physical computing device 100, and other considerations.

Based on the specific configuration of the virtual machine instances120, the control plane manager 150 can determine the virtual components140 that are associated with a virtual machine instance. The virtualmachine instances 120 may have different specifications associated withthe software and/or hardware of the virtual machine instance 120. Thedifferent specifications for the virtual machine instance 120 mayrequire specific virtual components 140, which may differ from thevirtual components configured for other virtual machine instances 120.

In some instances, the control plane manager 150 can receive a requestto launch a new instance. Based on the request, the control planemanager 150 can filter out physical computing devices 100 that cannothost the instance, such as physical computing devices that are full,physical computing devices that do not have the necessary hardware,physical computing devices that are already hosting too many instances,or physical computing devices that do not meet the requirements for thenew instance based on other reasons. The control plane manager canselect a physical computing device from the remaining physical computingdevices and sends a launch command to the offload device 130 or physicalcomputing device with configuration instructions for launching therequested instance having a specific configuration.

At (5), configuration instructions are sent to the offload device 130for configuration of the virtual components on the offload device 130.In some embodiments, the configuration instructions for the virtualmachine monitor are included with the offload device configurationinstructions. In addition, in some embodiments the configurationinstructions can be sent to the offload device 130 or a manager runningin Domain0 as part of an instance launch command. In this example, thecontrol plan manager 150 may have selected the physical computing device100 to host a virtual machine and sent a command to launch an instanceto the physical computing device 100.

At (6) the offload device can instantiate one or more virtual components140 on the offload device 130 based on the configuration instructionsreceived from the control plane manager. The virtual components 140 onthe offload device 130 represent virtual IO functions that areprovisioned for use by a virtual machine instance 120. Some non-limitingexamples of virtual components 140 that may be instantiated on anoffload device 130 include, a network interface controller (NIC), aprogrammable interrupt controller (PIC), a keyboard controller, an ISAbus, a floppy drive, a keyboard port, a mouse port, a monitor port, anda serial port. Some of the virtual components 140 instantiated for thevirtual machine instance 120 can be based on the configuration of thevirtual machine instance 120. Some of the virtual components 140 may berequired for operation of the virtual machine instance 120, regardlessof whether the virtual component 140 will be used. Some of the virtualcomponents 140 may be virtualizations of hardware components that existon the offload device, whereas others may be emulations of hardwarecomponents that do not exist on the offload device.

At (7), configuration instructions are sent to the physical computingdevice from the offload device 130 for configuration of the virtualmachine instances on the physical computing device 100. In someembodiments, the configuration instructions for the virtual machinemonitor 110 are sent directly from the control plane manager 150.

At (8), the virtual machine monitor 110 can instantiate the virtualmachine instances 120 based on the configuration instructions providedby the control plane manager 150 via the offload device 130. In someembodiments, the control plane manager 150 can communicate directly withthe virtual machine monitor 110. The virtual machine monitor 110provisions the logical virtualized resources that are associated withthe underlying physical resources to each virtual machine instance, suchas a VM processor 102 and VM memory 104 based on the configurationinstructions. The virtual machine monitor 110 can also provision storageresources that are included locally on the physical computing device 100or that are accessible via network.

In some embodiments, the virtual I/O components can be instantiated onthe physical computing device 100 and the offload device 130. In suchembodiments, a portion of the virtual I/O components are instantiated onthe physical computing device 100 and a portion of the virtual I/Ocomponents are instantiated on the offload device 130. The division ofthe virtual I/O components between the physical computing device 100 andthe offload device 130 can be determined by the configurationinstructions.

In some embodiments, the virtual machine monitor can include a memorymapping unit that manages the mapping of virtual components 140instantiated on the offload device 130 to the virtual machine instance120. The virtual machine monitor 110 can assign the virtual components140 to memory addresses of the offload device. The addresses mapped toeach virtual component 140 are provided to the virtual machine instance120 associated with virtual component 140. The virtual components 140associated with same virtual machine instance 120 may not besequentially arranged within the memory of the offload device. In someembodiments, the virtual machine monitor 110 can assign ranges of memoryaddresses on the offload device to each virtual machine instance 120.For example, if there were 12 virtual machine instances 120 the virtualmachine monitor 110 could assign separate ranges of memory addresses ofthe offload device 130 to each of the 12 virtual machine instances 120.

In other embodiments, the hosted virtual components 140 can beconfigured to communicate directly with a virtual machine instance 120.These virtual components 140 can be MMIO virtual components 140. In thisinstance the offload devices exposes or otherwise provides access to thevirtual components 140 directly to the virtual machine instance 120.Some virtual components 140 can be configured to communicate indirectlywith the virtual machine instance 120. For indirect communication, thevirtual machine instance 120 communicates with the virtual component viathe virtual machine monitor 110. The virtual machine monitor can createa translation table that is used to direct the IO communications betweenthe virtual machine instance 120 and the offload device 130. Differenttranslation tables can be created for port I/O virtual components andMMIO virtual components. The processes for direct and indirectcommunication between the virtual machine instances 120 and the virtualcomponents 140 are described in more detail with respect to FIGS. 3A and3B.

FIG. 2B illustrates a block diagram depicting the configuration ofvirtual machine instances 120 on the physical computing device 100 andvirtual components 140 on the offload device 130 by the virtual machinemonitor 110. The functions described in association with FIG. 2A can beperformed in a different sequence, can be added, merged, or left outaltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in certain embodiments,functions, acts or events can be performed concurrently.

At (1), the virtual machine monitor 110 can configure the physicalcomputing device 100 and the offload device 130. The virtual machinemonitor 110 runs on the hardware of the physical computing device 100and can communicate with the offload device 130. The virtual machinemonitor 110 can configure guest domains for each of the virtual machineinstances 120 on the physical computing device 100 and configure accessto virtual components 140 on the offload device 130.

The virtual machine monitor can configure any number of virtual machineinstances 120. The virtual machine instances 120 can be configuredautomatically based on defined criteria, instructions from othersystems, or other types of instructions. The virtual machine monitor canalso provide a management interface that can allow an administrator orother user to manually configure the virtual machine instances 120 onthe physical computing device 100.

With continued reference to FIG. 2B, as part of the configuration of thephysical computing device 100, the virtual machine monitor 110 candetermine the virtualized hardware resources allocated to each virtualmachine instance 120. A virtual machine instance 120 can have a specificconfiguration according to the computing capacity of the virtual machineinstance. This can be based on the requirements of a customer, thesystem, number of instances operating on a physical computing device100, and other considerations. The configuration information can includeconfiguring a translation table that is used for memory mapping to theI/O devices.

Based on the specific configuration of the virtual machine instances120, the virtual machine monitor 110 can determine the virtualcomponents 140 that are associated with a virtual machine instance. Thevirtual machine instances 120 may have different specificationsassociated with the software and/or hardware of the virtual machineinstance 120. The different specifications for the virtual machineinstance 120 may require specific virtual components 140, which maydiffer from the virtual components configured for other virtual machineinstances 120.

After the configuration of a virtual machine instance 120 is determined,at (2), the virtual machine monitor 110 can instantiate the virtualmachine instances 120. The virtual machine instances 120 provide adomain for operating systems and applications to run. The virtualmachine instances can be fully isolated from other virtual machineinstances 120. The virtual machine monitor 110 provisions the logicalvirtualized resources that are associated with the underlying physicalresources to each virtual machine instance, such as a VM processor 102and VM memory 104. The virtual machine monitor 110 can also provisionstorage resources that are included locally on the physical computingdevice 100, on the offload device 130, or that are accessible vianetwork. In combination with the provisioning of the virtual machineinstances 120, the virtual machine monitor is also responsible forprovisioning the 10 virtual components 140 that are required by thevirtual machine instance.

In some embodiments, the virtual I/O components can be instantiated onthe physical computing device 100 and the offload device 130. In suchembodiments, the virtual machine monitor 110 instantiates a portion ofthe virtual I/O components on the physical computing device 100 and aportion of the virtual I/O components on the offload device 130. Thedivision of the virtual I/O components between the physical computingdevice 100 and the offload device 130 can be determined by theconfiguration data associated with the virtual machine instance 120.

At (3) the virtual machine monitor 110 or a Domain-0 can cause theoffload device 130 to instantiate one or more virtual components 140 bysending information that identifies the type and umber of virtualcomponents to instantiate. The virtual components 140 on the offloaddevice 130 emulate functions performed by I/O physical components. Somenon-limiting examples of virtual components 140 that may be instantiatedon an offload device 130 include, a storage device, a network interfacecontroller (NIC), a programmable interrupt controller (PIC), a keyboardcontroller, an ISA bus, a floppy drive, a keyboard port, a mouse port, amonitor port, and a serial port. Some of the virtual components 140instantiated for the virtual machine instance 120 can be based on theconfiguration of the virtual machine instance 120. Some of the virtualcomponents 140 may be required for operation of the virtual machineinstance 120, regardless of whether the virtual component 140 will beused. Some of the virtual components 140 may be virtualizations ofhardware components that exist on the offload device, whereas others maybe virtualizations of hardware components that do not exist on theoffload device.

In some embodiments, the virtual machine monitor can include a memorymapping unit that manages the mapping of virtual components 140instantiated on the offload device 130 to the virtual machine instance120. The virtual machine monitor 110 can assign the virtual components140 to memory addresses of the offload device. The addresses mapped toeach virtual component 140 are provided to the virtual machine instance120 associated with virtual component 140. The virtual components 140associated with same virtual machine instance 120 may not besequentially arranged within the memory of the offload device. In someembodiments, the virtual machine monitor 110 can assign ranges of memoryaddresses on the offload device to each virtual machine instance 120.For example, if there were 12 virtual machine instances 120 the virtualmachine monitor 110 could assign separate ranges of memory addresses ofthe offload device 130 to each of the 12 virtual machine instances 120.

In other embodiments, the hosted virtual components 140 can beconfigured to communicate directly with a virtual machine instance 120.These virtual components 140 can be MMIO virtual components 140. In thisinstance the offload devices exposes or otherwise provides access to thevirtual components 140 directly to the virtual machine instance 120.Some virtual components 140 can be configured to communicate indirectlywith the virtual machine instance 120. For indirect communication, thevirtual machine instance 120 communicates with the virtual component viathe virtual machine monitor 110. The virtual machine monitor can createa translation table that is used to direct the IO communications betweenthe virtual machine instance 120 and the offload device 130. Differenttranslation tables can be created for port I/O virtual components andMMIO virtual components. The processes for direct and indirectcommunication between the virtual machine instances 120 and the virtualcomponents 140 are described in more detail with respect to FIGS. 3A and3B.

FIG. 3A illustrates the components of the virtual network depictingcommunication between a virtual machine instance 120 and a virtualcomponent 140 that uses the virtual machine monitor 110 as anintermediary between the virtual machine instance 120 and the virtualcomponent. The diagram illustrates indirect communication between thevirtual machine instance 120 and virtual component 140. Indirectcommunication can occur when a virtual component 140 virtualizes a PortI/O device and some MMIO devices. The devices can require communicationprotocols that do not allow the virtual machine instance 120 to directlyaccess the assigned memory addresses of the virtual components 140.

At (1) an I/O request from the virtual machine instance 120 triggers avirtual machine exit, also referred to as a VM exit. A VM exit is inresponse to certain I/O requests and/or events that marks the point atwhich a transition is made between the virtual machine instance 120currently running and the virtual machine monitor 110, which mustexercise system control to process the request. Virtual components 140that use indirect communication, and thus trigger VM exits, aredetermined during initialization and instantiation of the virtualmachine instance 120 and virtual components 140. The I/O requests thatcan trigger the VM exit are Port I/O requests and some identified MMIOrequests. The I/O request identifies a specific virtual function of avirtual component 140 and includes instructions for the identifiedvirtual component 140. When the VM exit is triggered by the I/O request,the virtual machine instance 120 sends the I/O request to the virtualmachine monitor 110.

At (2) the I/O request is received and translated by the virtual machinemonitor 110. The virtual machine monitor 110 receives the request anduses a translation table for translating the I/O request. Thetranslation table can include entries for each virtual component 140that requires a VM exit. The virtual machine monitor can includeseparate translation tables for different types of virtual components140. For example, Port I/O virtual components and MMIO virtualcomponents can have different translation tables. In some embodiments,the translation table can combine the translation information for PortIO virtual components 140 and MMIO virtual components 140 into a singletable. The configuration of the translation table can be preconfiguredby the virtual machine monitor. The translation table can store therouting information used for routing the received I/O request to theidentified virtual component 140. After the I/O request is received, thevirtual machine monitor can look up the I/O request in the translationtable and route the request to the memory address of the offload device130 that is responsive to the I/O request.

At (3), the I/O request is routed to the identified virtual component140B. The I/O request is sent from the physical computing device 100over the interface to the offload device 130. At (4) the I/O request isreceived and resolved by the virtual component 140B. The virtualcomponent 140B can resolve the request based on the informationcontained in the I/O request. The virtual component 140 can resolve therequest based on the virtual function that is assigned to the memoryaddress identified in the I/O request. The processing of the request isperformed by the computing resources of the offload device and does notutilize the computing resources assigned to the virtual machine instance120. The I/O request can be a simple read or write, or a complexalgorithm that is implemented on the offload device. For example, theoffload device 130 may execute one or more device emulator programs. Theoffload device 130 may process the request and identify the appropriateemulator based on characteristics of the request. Next, the deviceemulator can run and process the request. For virtual components 140 setup on the offload device, the type of request is not considered whendetermining whether route the request using the offload device. Rather,the virtual machine monitor 110 routes the request regardless of theoperations that are to be performed based on the request.

Based on the type of the request, the resolution of the requestperformed by the virtual component 140 can differ. In some instances,the I/O request may require a response from the virtual component 140B.In some instances, the I/O request is resolved and may send anacknowledgment back to the virtual machine instance. For example, awrite command from the virtual machine instance 120 to the virtualdevice may require a response from the virtual component 140 to thevirtual machine instance 120. In which case, an acknowledgment can besent from the virtual component 140 to the virtual machine instance 120.

In some instances, the process continues when an I/O request requires aresponse from the virtual component 140 to the virtual machine instance,such as a read request. In which case, after the request is resolved thevirtual component 140 responds to the request. At (5), the virtualcomponent 140 sends a response to the I/O request. The response caninclude the information specific to the request and is dependent on thespecific parameters of the I/O request and the virtual component, or maybe an acknowledgement.

At (6), the virtual machine monitor 110 translates that response to theI/O request. The translation of the response is performed using thetranslation table to route the response to the virtual machine instance120 that initiated the request. At (7), the response to the I/O requestis sent to the identified virtual machine instance 120 from the virtualmachine monitor based on the information stored in the translation tableof virtual machine monitor. The response can be accompanied by a VMresume that closes out the VM exit. At this point, the process completesand can be reinitiated each time an IO request is required that uses thetranslation table of the virtual machine monitor to act as anintermediary between the virtual machine instance 120 and the virtualcomponent.

FIG. 3B illustrates a block diagram depicting direct communicationbetween a virtual machine instance 120 and a virtual component. Virtualmachine instances 120 can communicate directly with the MMIO virtualcomponents. The MMIO virtual components 140 allow for directcommunication between the virtual machine instance 120 and the memoryregisters assigned to the virtual component 140 without any interactionbetween the virtual machine instance 120 and the virtual machine monitor110.

At (1) the virtual machine transmits an I/O request to a virtualcomponent. The virtual machine instance 120 transfers the request to thevirtual component 140 using memory addressing assigned to the virtualcomponent 140 during instantiation of the virtual machine instance 120and the virtual component. The virtual component 140 can be assigned toa range of memory addresses for communication with the virtual machineinstance. Depending on the type of request the virtual machine instance120 can communicate with a specific memory register. The memory mappingof the virtual component 140 allows the virtual machine instance 120 tocommunicate directly with the virtual component 140 through theinterconnect interface 106. A memory management unit can translatedevice-visible virtual memory addresses to physical memory addresses onthe offload device 130.

At (2), the I/O request is resolved by the device. The I/O request canbe any type of request such as a read or write request. The virtualcomponent 140 can resolve the request based on the specific parametersof the request. The processing of the request is performed by thecomputing resources of the offload device and does not utilize thecomputing resources assigned to the virtual machine instance 120. TheI/O request can be a simple read or write, or a complex algorithm thatis implemented on the offload device. For virtual components 140 set upon the offload device, the type of request is not considered whendetermining whether route the request using the offload device. Rather,the virtual machine monitor 110 routes the request regardless of theoperations that are to be performed based on the request.

At (3), the virtual component 140 can generate a response and send it tothe virtual machine instance. For example, an acknowledgment can be sentfrom the virtual component 140 to the virtual machine instance 120.

FIG. 4 illustrates a flow diagram of a routine 400 depicting theconfiguration of a virtual environment on a physical computing device100 and an offload device 130. The steps of the routine 400 are beingdescribed as generally being performed by a control plane manager 150.The functions described in association with FIG. 4 can be performed in adifferent sequence, can be added, merged, or left out altogether (e.g.,not all described acts or events are necessary for the practice of thealgorithms). Moreover, in certain embodiments, functions, acts or eventscan be performed concurrently.

At block 402, the control plane manager 150 receives resourceinformation associated with the physical computing device and theoffload device. The resource information can include information such asthe available computing resources, such as processor type, memoryconfiguration, hardware component configuration, versioning information,and other information that identifies the resources of the physicalcomputing device and the offload device. The resource information caninclude current operating conditions, computer resource utilizationinformation associated with the current configuration of the physicalcomputing device 100 and the offload device. The resource informationcan be gathered based on a request provided by the offload device 130 orthe control plane manager 150.

As part of a request to launch a virtual machine, and as illustrated atblock 404, the control plane manager can determine a configuration for avirtual machine instance to launch on the physical computing device 100.The determination can be based, in part, on the resource information andinformation independent of the resource information. For example, thecontrol plane manager 150 can also base the configuration on otherconsiderations, such as client specifications, the configurations ofother physical computing devices 100, such as clustered computingdevices, or other considerations independent of the resource informationassociated with the physical computing device.

As part of determining the configuration of the virtual machine, thecontrol plane manager 150 can determine the virtualized hardwareresources that will need to be allocated to the virtual machine instance120. A virtual machine instance 120 can have a specific configurationaccording to the computing capacity of the virtual machine instance.This can be based on the resource information, requirements of acustomer, the system, the number of instances operating on the physicalcomputing device 100, and other considerations. The virtual machineinstances 120 may have different specifications associated with thesoftware and/or hardware of the virtual machine instance 120. Thedifferent specifications for the virtual machine instance 120 mayrequire specific virtual components 140, which may differ from thevirtual components configured for other virtual machine instances 120.

At block 406, the configuration instructions are provided to the offloaddevice 130 for configuration of the virtual components on the offloaddevice 130. The offload device can instantiate one or more virtualcomponents 140 on the offload device 130 based on the configurationinstructions received from the control plane manager. Based on thespecific configuration of the virtual machine instances 120, the offloaddevice 130 and/or the virtual machine monitor can determine the virtualcomponents 140 to instantiate on the offload device 130.

At block 408, configuration instructions are provided to the physicalcomputing device from the offload device 130 for configuration of thevirtual machine instances on the physical computing device 100. In someembodiments, the configuration instructions for the virtual machinemonitor 110 are sent directly from the control plane manager 150 to thevirtual machine monitor. The virtual machine monitor 110 can instantiatethe virtual machine instances 120 based on the configurationinstructions provided by the control plane manager via the offloaddevice. In some embodiments, functions associated with blocks 406 and408 can occur substantially simultaneously in response to a instancelaunch request.

FIG. 5 illustrates a flow diagram of a routine 500 depicting theconfiguration of a virtual environment on a physical computing device100 and an offload device 130. The steps of the routine 500 are beingdescribed as generally being performed by a virtual machine monitor 110on the physical computing device 100 and/or by management software onthe offload device 130, such as the management module 138 and deviceemulation module 139.

At block 502 the virtual machine monitor 110 and management softwarerunning on offload device 130 boot on the physical computing device 100.The virtual machine monitor 110 can instantiate instances on thephysical computing device 100. The virtual machine monitor 110 cancontrol the provisioning of resources of the hardware resources from themachine to specific virtual machine instances 120 and those virtualmachine instances 120 can then be logically associated with theunderlying hardware resources.

After a request to launch an instance is received, and as illustrated atblock 504, the virtual machine monitor 110 can determine theconfiguration of the virtual machine instances 120. The determination ofthe virtual machine instances 120 can be based on information providedfrom an administrator or control plane manager 150. The configurationcan be a default configuration based on defined operating proceduresstored within the virtual machine monitor 110 or it can be passed to theoffload device 130 from the control plane manager 150. In some instancesthe virtual machine monitor may be manually controlled by a user inorder to determine the configuration for the virtual machine instances120. The determination of the configuration of the virtual machineinstances 120 can include the number of virtual machine instances 120,the resources provisioned to each virtual machine instance, one or moremachine image to use to generate each virtual machine instance, andother virtual machine instance 120 configuration information. The numberof virtual machine instances 120 can be based on the computing resourcesof the physical computing device 100, the offload device 130, or otherconfiguration parameter. The number of virtual machine instances 120 maynot limited by any predefined limit. The provisioning of virtualcomputing resources assigned to each virtual machine instance 120 can bebased on logical physical resources of the physical computing device100. The specific configuration of the virtualized computing resourcescan vary for each virtual machine instance. The virtualized computingresources can include a virtual machine CPU, virtual machine memory andother resources used for the operation of the virtual machine instance.The machine image used to instantiate a virtual machine instance caninclude the operating system (e.g., Windows®, Linux®, etc.),applications, and any other software. The type of virtual components 140required by the virtual machine instance 120 can be based on thespecific configuration of the virtual machine instance. The virtualmachine monitor 110 can determine the allocation of virtual components140 between the physical computing device 100 and the offload device130. The virtual machine instance 120 may be configured with somevirtual components 140, such as data stores, and the offload device 130may be configured with some virtual components 140. In some embodiments,the virtual components 140 may be allocated primarily to the offloaddevice 130.

At block 506, the virtual machine monitor can instantiate the virtualmachine instances 120 on the physical computing device 100. Each virtualmachine instance 120 is configured within a guest domain on the physicalcomputing device 100. Each guest domain is configured to be independentof the other guest domains. The instantiation of the virtual machineinstance 120 includes the determined configuration of computingresources, software, and other components as determined at block 404. Inan example embodiment, the virtual machine monitor can determine theallocation of the virtual components 140 between the physical computingdevice 100 and the offload device 130. For example, the virtual machinemonitor may configure the physical computing device 100 to not have anyvirtual components 140 allocated to the physical computing device 100and all of the virtual components 140 allocated to the offload device130. Alternatively, the management module 138 on the offload device 130can determine the allocation of virtual components 140 between thephysical computing device 100 and the offload device 130.

At block 508, the virtual components 140 can be configured on theoffload device 130. For example, either the virtual machine monitor, adomain-0 management program, or management programs on the offloaddevice 130, such as the management module 138 and the device emulationmodule 139, can configure the virtual components 140. The number andtype of virtual components 140 on the offload device 130 can be based onthe specific configuration of the virtual machine instance. The virtualcomponents 140 can include MMIO virtual components 140 and Port IOvirtual components 140. The virtual components 140 on the offload device130 can be logically partitioned according to their associated virtualmachine instance 120. The virtual components 140 are associated withvirtual functions. The virtual components 140 are instantiated in thememory of the offload device 130. In some instances the offload device130 can have defined partitions that that include sequential ranges ofmemory assigned to a virtual machine instance. In some embodiments, thevirtual components 140 on the offload device 130 are assigned to logicallocations within the memory, which may not be sequential in nature. Theinstantiation of the virtual components 140 on the offload device 130allows for the physical computing device 100 to not have to allocatememory to the instantiation of the virtual components 140. Some of thevirtual components 140 may never be using in a virtual operatingenvironment but are required for the operation of the virtual machineinstance. The instantiation of the virtual components 140 can result inresource usage overhead that reduces the available computing resourcesthat are provisioned to the virtual machine instances 120. By allocatingthe virtual components 140 on the offload device 130, computingresources on the physical computing device 100 can be freed up for usageby the virtual machine instances 120.

At block 510, the virtual machine monitor can determine the memorymapping of the virtual components 140 and the communication between thevirtual components 140 and the virtual machine instance. A memorymanager unit can determine the memory mapping for enabling communicationbetween the virtual machine instances and the virtual components 140.Depending on the type of virtual component, the virtual machine monitorcan allow for direct access to the virtual component 140 or can provideindirect access via the virtual machine monitor. For direct access thevirtual machine monitor assigns addresses to the virtual components 140that allow the virtual machine instance 120 to communicate directly withthe virtual components. For indirect access, the virtual machine monitor110 can configure a translation table for communication between thevirtual machine instances 120 and the virtual components 140. Thetranslation table can include addressing information for port I/Ovirtual components 140 and some MMIO virtual components 140.

The instantiation of the virtual components 140 on the offload device130 can be done in parallel with the instantiation of the virtualmachine instance 120 on the physical computing device 100. Some stepscan be done in parallel while others may be done sequentially. The stepsare merely illustrative of logical processes that are being performed bythe virtual machine monitor on the physical computing device 100 and theoffload device 130.

FIG. 6 is a flow diagram illustrating a routine 600 depicting indirectcommunication between the host computing device and the virtual devicevia the virtual machine monitor 110. The steps of the routine 600 arebeing described as generally being performed by a virtual machinemonitor 110 on the physical computing device 100.

At block 602 the virtual machine monitor receives an I/O request from avirtual machine instance 120, which is triggered by a VM exit. When theVM exit is triggered by the I/O request, the virtual machine instance120 sends the I/O request to the virtual machine monitor. The I/Orequest comes from the virtual machine instance 120 for a specificvirtual component.

At block 604, the I/O request is translated by the virtual machinemonitor 110. The virtual machine monitor receives the request and has atranslation table that is used for translating the I/O request. Thetranslation table can include entries for each virtual component 140that requires a VM exit. The virtual machine monitor can includeseparate translation tables for different types of I/O devices. Forexample, Port IO virtual components 140 and MMIO virtual components 140can have different translation tables. In some embodiments, thetranslation table can combine the translation information for Port IOvirtual components 140 and MMIO virtual components 140. Theconfiguration of the translation table can be preconfigured by thevirtual machine monitor. The translation table can be a lookup tablethat can store the routing information for the virtual components 140.When the I/O request is received, the virtual machine monitor can lookup the I/O request in the translation table and determine the routinginformation for directing the I/O request to the correct locationassociated with the addressed virtual component on the offload device130.

At block 606, the virtual machine monitor 110 routes the I/O request tothe identified virtual component. The I/O request is sent from thephysical computing device 100 over the interface bus to the offloaddevice 130. The processing of the request is performed by the computingresources of the offload device and does not utilize the computingresources assigned to the virtual machine instance 120. The I/O requestcan be a simple read or write, or a complex algorithm that isimplemented on the offload device. For virtual components 140 set up onthe offload device, the type of request is not considered whendetermining whether route the request using the offload device. Rather,the virtual machine monitor 110 routes the request regardless of theoperations that are to be performed based on the request.

At block 608, the virtual machine monitor receives a response to the I/Orequest from the virtual component. When a response is not required, thevirtual machine monitor 110 can receive an acknowledgment from thevirtual component 140. The response can include the informationresponsive to the request, which can be dependent on the specificparameters of the IO request and the virtual component.

At block 610, the virtual machine monitor 110 translates that responseto the I/O request. The translation of the response is performed usingthe translation table to route the response to the virtual machineinstance 120 that initiated the request.

At block 612, the response to the I/O request is sent to the identifiedvirtual machine instance 120 from the virtual machine monitor based onthe information stored in the translation table of virtual machinemonitor. The response can be accompanied by a VM resume that closes outthe VM exit. At this point the process completes and can be reinitiatedeach time an IO request is required that uses the translation table ofthe virtual machine monitor to act as an intermediary between thevirtual machine instance 120 and the virtual component.

FIGS. 7A, 7B, 7C and 8 are block diagrams illustrating interactionsbetween the physical computing device 100 and offload device 130. Theprocesses illustrated in FIGS. 7A, 7B, 7C and 8 can be performed inassociation with any number of virtual machine instances andcorresponding virtual components. However, for simplicity only a singlevirtual machine instance 120 and a single virtual component 140 isillustrated.

A virtual machine monitor 110 may coordinate certain communications andprocesses of the virtual machine instances 120, as discussed above inrelation to FIGS. 1-6. In an illustrative embodiment, the physicalcomputing device 100 may reserve a portion of memory 124 used to storestate configuration information and I/O request identificationinformation associated with the virtual components. The memory on thephysical computing device can be volatile memory.

The offload device 130 includes virtual components 140 associated with avirtual machine instance 120. The virtual components 140 can beinitialized and configured for operation during the booting process ofthe virtual machine instance 120 in order to resolve requests made byvirtual machine instances 120 hosted by the physical computing device,100.

The virtual machine instance 120 can include a virtual componentmanagement module that can be configured to perform aspects of managingvirtual component checkpoints. The virtual component management module120 may provide instructions that control a process of storing andupdating state configuration information associated with the virtualcomponents. The virtual component management module 120 may also provideinstructions that control a process of storing and updating I/O requestidentification information. In some embodiments, the virtual componentmanagement module can be involved in the recovery of the virtualcomponents after a triggering event.

The state configuration information can include virtual componentconfiguration information that may be used by a virtual component 140 torecreate an operational state of the virtual component. In one example,the virtual component configuration information can include informationthat designates a value of variety of variables that may be used asinputs for determination of a configuration of a virtual component. Inanother example, the virtual component configuration information caninclude indicators reflective of a state of a virtual component (such asan indicator that a virtual component has recently been initialized oris running in a particular mode, such as a safe mode). However, thevirtual component configuration information can also include othervirtual component configuration information. For example, the stateconfiguration information associated with the operational state of avirtual component may include a value that dictates the manner in whicha virtual component may perform a service in response to an I/O request,the amount of offload device resources allowed to the virtual componentrelative to any other virtual component, or any other aspect of avirtual component.

In some embodiments, the virtual component state configurationinformation may be stored in accordance with a rule set that determinesthe virtual component state configuration information that may be storedin different situations in order to reconfigure the operational state ofthe virtual component, such as where different sets of virtual componentstate configuration information may be saved for different virtualcomponents, for different virtual components in different states, fordifferent virtual machine instances using the virtual component or asbased on any other criteria. Furthermore, the virtual component stateconfiguration information may be stored using a virtual componentidentifier that can be used to determine the location of virtualcomponent state information in memory on the physical computing device.

The I/O request identification information can include information thatcan be used to identify I/O requests pending resolution by the virtualcomponent. The I/O requests may be for virtual I/O functions performedby a virtual component The I/O request identification information may bea value representative of an I/O request that may be recognizable by avirtual machine instance that issued the I/O request for resolution by avirtual component. In certain embodiments, the I/O requestidentification information may also include a quantity of I/O requestspending resolution by a particular virtual component. In someembodiments, the I/O request identification information may characterizean I/O request that has not yet been resolved. In certain embodiments,the I/O request identification information may be stored in accordancewith an I/O request identification information rule set that determinesthe I/O request identification information that may be stored indifferent situations. For example, such as where I/O requestidentification information may be saved differently for different I/Orequests, for different virtual components resolving the I/O request,for different virtual machine instances using the virtual component toresolve the I/O request or as based on any other criteria. Furthermore,the I/O request identification information may be stored using a virtualcomponent identifier that can be used to determine the location of theI/O request identification information in memory on the physicalcomputing device.

FIG. 7A is a block diagram illustrating a process of storing virtualcomponent state configuration information defining an initialoperational state configuration of the virtual component for a virtualcomponent checkpoint.

At (1), during the boot process the virtual component can be identifiedand initialized for the resolution of various I/O requests based on aninitialization instruction sent from the virtual machine instance to theoffload device 130. The initialization instruction can initialize andconfigure the virtual component with an initial state configuration.

At (2), upon initialization, the virtual component 140 sends stateconfiguration information defining the initial state configuration ofthe virtual component. The configuration information can include astorage instruction for storing the state configuration in the memory124 of the physical computing device 100.

At (3), the memory 124 stores the state configuration informationdefining the initial state configuration of the virtual component. Thestorage of the state configuration information can be based upon astorage instruction received from the virtual component 140. Althoughthe illustrated embodiment describes the virtual component 140 astransmitting the initial virtual component state configuration storageinstruction to the memory 124, the initial virtual component stateconfiguration storage instruction may be sent by a virtual componentmanagement module to the memory in addition to and/or instead of thevirtual component 140 doing so in accordance with different embodiments.

FIG. 7B is a block diagram illustrating a process of storing virtualcomponent state configuration information defining an updated stateconfiguration of the virtual component 140.

At (1), the virtual machine instance 120 sends a virtual componentupdate instruction to the virtual component 140. The virtual componentupdate instruction is configured to update at least one parameter of thevirtual component, thereby causing a change to the state configurationof the virtual component 140.

At (2), the configuration of the virtual component is updated based onthe virtual component update instruction received from the virtualmachine instance. At (3), the virtual component 140 sends a virtualcomponent state configuration update storage instruction to the memory124 of the physical computing device 100 The storage instructioncomprises the state configuration data for storage in the memory 124.

At (4), the memory 124 is configured to update the virtual componentstate configuration information based upon the virtual component stateconfiguration update storage instruction to reflect the updatedconfiguration of the virtual component 140 in accordance with thevirtual component update instruction. Although the illustratedembodiment describes the virtual component 140 as transmitting thevirtual component state configuration update storage instruction to thememory 124, the virtual component state configuration update storageinstruction may be sent by the virtual component management module tothe memory in addition to and/or instead of the virtual component 140doing so in accordance with different embodiments.

FIG. 7C is a block diagram illustrating a process of storing I/O requestidentification information. At (1), the virtual component managementmodule of the virtual machine instance 120 sends an I/O request forresolution by the virtual component 140. At (2), the virtual component140 sends a storage instruction including I/O request identificationinformation associated with the I/O request to the memory 124 of thephysical computing device 100. The I/O request identificationinformation comprises identification information sufficient for thevirtual machine instance to identify the I/O request. Although theillustrated embodiment describes the virtual component 140 astransmitting the I/O request identification storage instruction to thememory 124, the I/O request identification storage instruction may besent by the virtual component management module to the memory inaddition to and/or instead of the virtual component 140 doing so inaccordance with different embodiments. At (3), the memory 124 of thephysical computing device 100 stores I/O request identificationinformation that identifies the I/O request.

At (4), the virtual component 140 resolves the I/O request. At (5), thevirtual component 140 provides a response to the I/O request back to thevirtual machine instance 120.

At (6), the virtual component 140 sends an I/O request updateinstruction to the memory 124. Although the illustrated embodimentdescribes the virtual component 140 as transmitting the I/O requestupdate instruction to the memory 124, the I/O request update instructionmay be sent by the virtual component management module to the memory inaddition to and/or instead of the virtual component 140 doing so inaccordance with different embodiments. At (7), the memory 124 updatesthe I/O request identification information based on the I/O requestupdate instruction to reflect that the I/O request has been resolved.

FIG. 8 is a block diagram illustrating a process of retrieving virtualcomponent state configuration information and I/O request identificationinformation. At (1), the virtual component 140 receives a triggeringevent. The triggering event can be caused by a hardware crash, softwarecrash, network based command or other event affecting the offloaddevice. In some instances, The triggering event can cause the offloaddevice to need to be rebooted in order to resolve the triggering event.In some instances, only the affected virtual components will need to berebooted.

At (2), the virtual component 140 sends a notification of the triggeringevent including a request to retrieve the stored virtual component stateconfiguration information for the virtual component 140 and any storedI/O request identification information. Although the illustratedembodiment describes the virtual component 140 as transmitting thetriggering event notification directly to the memory 124, the triggeringevent notification may be sent by the virtual component 120 to thevirtual component management module, which instructs the virtualcomponent to retrieve the stored virtual component state configurationinformation and I/O request identification information from the memory124 in accordance with different embodiments.

At (3), the memory 124 retrieves the stored virtual component stateconfiguration information as well as the I/O request identificationinformation for the virtual component 140. In certain embodiments, thereboot notification may include identifiers associated with therequesting virtual component that may be cross correlated with thestored virtual component state configuration information and the I/Orequest identification information for the specific requesting virtualcomponent as stored in a look up table or other data store in thememory.

At (4), the memory 124 sends the stored virtual component stateconfiguration information as well as the I/O request identificationinformation for the virtual component 140 to the virtual component 140.

At (5), the virtual component 140 is reconfigured in accordance with thevirtual component state configuration information. As discussed above,the virtual component state configuration information may be anyinformation that may be used by a virtual component to recreate avirtual component state configuration, such as information thatdesignates a value of variety of variables that may be used as inputsfor determination of a configuration of a virtual component and/orindicators reflective of a state of a virtual component. Thereby, usingthe virtual component state configuration information, the virtualcomponent may revert back to its operational state configuration justprior to the receipt of the triggering event. Also, as discussed above,the I/O request identification information stores transactionidentifiers for I/O requests that the virtual component received beforethe triggering event, but were not resolved.

At (6), the virtual component 140 sends a resend request to the virtualcomponent management module of the virtual machine instance 120 for theidentified I/O requests as identified from the I/O requestidentification information. At (7), the virtual component 140 resolvesthe resent I/O requests and, at (8), responds to the resent I/Orequests.

FIG. 9 is a flow diagram illustrating a routine 900 for storing virtualcomponent state configuration information and I/O request identificationinformation. At block 902, the initial virtual component stateconfiguration is stored as part of the virtual component stateconfiguration information. As discussed above, the virtual componentstate configuration information may be stored in the memory 124 of thephysical computing device 100 based on a virtual component stateconfiguration initial storage instruction sent by the virtual component140.

At block 904, a decision is made as to whether the virtual componentstate configuration has been updated. As discussed above, the virtualcomponent state configuration may be updated based upon a virtualcomponent update instruction received by the virtual component 140 fromthe virtual component management module of the virtual machine instance120.

At block 906, the virtual component state configuration is updated. Thevirtual component state configuration may be updated in accordance withthe virtual component update instruction sent from the virtual component140 to the memory 124.

At block 908, an I/O request is received by the virtual component 140from the virtual machine instance 120. The I/O request may be receivedby the virtual component 140 for resolution of the I/O request by avirtual I/O function of the virtual component 140.

At block 910, I/O request identification information that identifies theI/O request is stored in the memory of the physical computing device.The I/O request identification information may be stored in accordancewith an I/O request identification storage instruction sent by thevirtual component 140 to the volatile memory 124.

At block 912, the I/O request identification information is updated toreflect resolution of the I/O request by the virtual component 140. Incertain embodiments, the I/O request information may be updated by thevirtual component 140 sending an I/O request update instruction from thevirtual component 140 to the memory 124 to delete the initial I/Orequest identification information identifying the resolved I/O request.

FIG. 10 is a flow diagram illustrating a routine 1000 for retrievingvirtual component state configuration information and I/O requestidentification information after the occurrence of a triggering event.The virtual component may receive a triggering event based on variousevents, such as due to a hardware crash, software crash, network basedcommand, or other defined event.

At block 1002, the virtual component receives a triggering event. Thetriggering event may be received based on a various events, such as dueto a hardware crash, software crash, network based command or otherevent affecting the offload device. As discussed above, although thevarious processes may be described in the context of the virtualcomponent 140, the memory or the virtual component management module ofthe virtual machine instance 120, each of these processes are performedbased upon instructions from the virtual component management modulethat configures the virtual component 140, the memory 124 or the virtualmachine instance 120 to perform these various processes.

At block 1004, the virtual component 140 retrieves virtual componentstate configuration information and I/O request identificationinformation from the memory 124 of the physical computing device 100that the offload device 130 hosting the virtual component 140 isinterfaced with. As discussed above, the virtual component stateconfiguration information and I/O request identification information maybe previously stored as discussed in connection with FIG. 8.

At block 1006, a decision is made as to whether there are any pendingI/O requests. This decision is made with regards to whether any I/Orequests are received at the virtual component 140 by the virtualcomponent management module of the virtual machine instance 120 hostedby the physical computing device 100 as determined by the retrieved I/Orequest identification information. If there are no pending I/Orequests, the process proceeds through block A to block 904 of FIG. 9,where a decision is made as to whether the virtual component stateconfiguration has been updated.

At block 1008, a resend request for identified I/O request(s) from theI/O request identification information is sent from the virtualcomponent 140 to the virtual component management module of the virtualmachine instance 120.

At block 1010, the virtual component 140 receives the resent I/Orequests in accordance with the resend request for identified I/Orequests.

At block 1012, the I/O request identification information is updatedafter the I/O request is resolved and responded to by the virtualcomponent 140. As discussed above, the I/O request information may beupdated by the virtual component 140 sending an I/O request updateinstruction from the virtual component 140 to the memory 124 to deletethe initial I/O request identification information identifying theresolved I/O request. After the I/O request identification is updated,the process proceeds through block A to block 904 of FIG. 9, where adecision is made as to whether the virtual component state configurationhas been updated.

It is to be understood that not necessarily all objects or advantagesmay be achieved in accordance with any particular embodiment describedherein. Thus, for example, those skilled in the art will recognize thatcertain embodiments may be configured to operate in a manner thatachieves or optimizes one advantage or group of advantages as taughtherein without necessarily achieving other objects or advantages as maybe taught or suggested herein.

All of the processes described herein may be embodied in, and fullyautomated via, software code modules executed by a computing system thatincludes one or more general purpose computers or processors. The codemodules may be stored in any type of non-transitory computer-readablemedium or other computer storage device. Some or all the methods mayalternatively be embodied in specialized computer hardware. In addition,the components referred to herein may be implemented in hardware,software, firmware or a combination thereof.

Many other variations than those described herein will be apparent fromthis disclosure. For example, depending on the embodiment, certain acts,events, or functions of any of the algorithms described herein can beperformed in a different sequence, can be added, merged, or left outaltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in certain embodiments, acts orevents can be performed concurrently, e.g., through multi-threadedprocessing, interrupt processing, or multiple processors or processorcores or on other parallel architectures, rather than sequentially. Inaddition, different tasks or processes can be performed by differentmachines and/or computing systems that can function together.

The various illustrative logical blocks, modules, and algorithm elementsdescribed in connection with the embodiments disclosed herein can beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, and elementshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. The described functionality can be implemented invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A general purpose processor can be a microprocessor,but in the alternative, the processor can be a controller,microcontroller, or state machine, combinations of the same, or thelike. A processor can include electrical circuitry configured to processcomputer-executable instructions. In another embodiment, a processorincludes an FPGA or other programmable device that performs logicoperations without processing computer-executable instructions. Aprocessor can also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Although described hereinprimarily with respect to digital technology, a processor may alsoinclude primarily analog components. For example, some or all of thesignal processing algorithms described herein may be implemented inanalog circuitry or mixed analog and digital circuitry. A computingenvironment can include any type of computer system, including, but notlimited to, a computer system based on a microprocessor, a mainframecomputer, a digital signal processor, a portable computing device, adevice controller, or a computational engine within an appliance, toname a few.

The elements of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module stored in one or more memory devices andexecuted by one or more processors, or in a combination of the two. Asoftware module can reside in RAM memory, flash memory, ROM memory,EPROM memory, EEPROM memory, registers, hard disk, a removable disk, aCD ROM, or any other form of non-transitory computer-readable storagemedium, media, or physical computer storage known in the art. An examplestorage medium can be coupled to the processor such that the processorcan read information from, and write information to, the storage medium.In the alternative, the storage medium can be integral to the processor.The storage medium can be volatile or nonvolatile. The processor and thestorage medium can reside in an ASIC. The ASIC can reside in a userterminal. In the alternative, the processor and the storage medium canreside as discrete components in a user terminal.

Conditional language such as, among others, “can,” “could,” “might” or“may,” unless specifically stated otherwise, are otherwise understoodwithin the context as used in general to convey that certain embodimentsinclude, while other embodiments do not include, certain features,elements and/or steps. Thus, such conditional language is not generallyintended to imply that features, elements and/or steps are in any wayrequired for one or more embodiments or that one or more embodimentsnecessarily include logic for deciding, with or without user input orprompting, whether these features, elements and/or steps are included orare to be performed in any particular embodiment.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each is present.

Any process descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or elements in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown, or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved as would be understood by those skilled in the art.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

1-20. (canceled)
 21. A computing device comprising: one or moreprocessors configured to execute computer-readable instructions that,upon execution, configure the computing device to: execute one or morevirtual input/output (I/O) components, wherein the one or more virtualI/O components are associated with a virtual machine instance hosted ona server computing device; detect a change in state of a virtual I/Ocomponent of the one or more virtual I/O components; after detecting thechange in state of the virtual I/O component, execute a write to memoryof the server computing system to store virtual component state datacomprising at least the change in state of the virtual I/O component inthe memory of the server computing device; and after an error thatimpacts operation of the virtual I/O component, use the virtualcomponent state data stored in the memory of the server computer torestore at least a portion of state of the virtual I/O component. 22.The computing device of claim 21, wherein the computer-readableinstructions further configure the computing device to: use the virtualcomponent state data stored in memory after a reboot of the computingdevice.
 23. The computing device of claim 21, wherein execution of thewrite to memory includes providing a state configuration storageinstruction associated with the virtual I/O component to the servercomputing device.
 24. The computing device of claim 22, wherein thecomputer-readable instructions further configure the computing device tosend a request for pending I/O requests for resolution by the virtualI/O component when restoring at least a portion of state of the virtualI/O component.
 25. The computing device of claim 21, wherein the virtualmachine instance is one of a plurality of virtual machine instanceshosted by the server computing device.
 26. The computing device of claim21, wherein the virtual I/O component is one of a plurality of virtualI/O components hosted by the computing device.
 27. The computing deviceof claim 21, wherein the server computing device is configured to storethe virtual component state data in volatile memory.
 28. A computerimplemented method comprising: by a computing device, executing one ormore virtual input/output (I/O) components, wherein the one or morevirtual I/O components are associated with a virtual machine instancehosted on a server computing device; detecting a change in state of avirtual I/O component of the one or more virtual I/O components; afterdetecting the change in state of the virtual I/O component, executing awrite to memory of the server computing system to store virtualcomponent state data comprising at least the change in state of thevirtual I/O component in the memory of the server computing device; andafter an error that impacts operation of the virtual I/O component,using the virtual component state data stored in the memory of theserver computer to restore at least a portion of state of the virtualI/O component.
 29. The method of claim 28 further comprising receiving avirtual I/O component initialization instruction to configure an initialstate of the virtual I/O component.
 30. The method of claim 28 furthercomprising detecting the change in state of the virtual I/O componentperiodically.
 31. The method of claim 28, wherein the change in state ofthe virtual I/O component is an I/O request pending for resolution bythe virtual I/O component.
 32. The method of claim 31, wherein restoringat least a portion of the state further comprising sending a request forI/O requests to the server computing device based, at least in part, onI/O request identification data retrieved from the memory of the servercomputing device.
 33. The method of claim 31, further comprisingresolving, by the virtual I/O component, the I/O request, and providingan I/O request resolution instruction associated with the I/O request tothe server computing device, wherein the server computing device isconfigured to update the virtual component state data based onresolution of the request.
 34. The method of claim 28, wherein the I/Orequest is resolved using computing resources of the computing device.35. The method of claim 28, wherein the computing device is operativelycoupled to the server computing device.
 36. A non-transitorycomputer-readable storage medium having stored thereon computer-readableinstructions that, when executed, configure a computing system to:execute one or more virtual input/output (I/O) components, wherein theone or more virtual I/O components are associated with a virtual machineinstance hosted on a server computing device; detect a change in stateof a virtual I/O component of the one or more virtual I/O components;after detecting the change in state of the virtual I/O component,execute a write to memory of the server computing system to storevirtual component state data comprising at least the change in state ofthe virtual I/O component in the memory of the server computing device;and after a triggering event impacts operation of the virtual I/Ocomponent, use the virtual component state data stored in the memory ofthe server computer to restore at least a portion of state of thevirtual I/O component.
 37. The non-transitory computer-readable storagemedium of claim 36, wherein the computer-readable instructions furtherconfigure the computing device to reboot a computing device based on thereceipt of the triggering event.
 38. The non-transitorycomputer-readable storage medium of claim 36, wherein thecomputer-readable instructions further configure the computing device toinstantiate the virtual I/O component based, at least in part, on thevirtual component state data such that the virtual I/O device is in thesame state as prior to the triggering event.
 39. The non-transitorycomputer-readable storage medium of claim 36, wherein the change instate is an I/O request pending for resolution by the virtual I/Ocomponent.
 40. The non-transitory computer-readable storage medium ofclaim 36, wherein the computer-readable instructions further configurethe computing device to send a request for I/O requests to the servercomputing device based, at least in part, on I/O request identificationdata retrieved from the memory of the server computing device whenrestoring at least a portion of the state.